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Psychometric theory has been one of psychology's 
stronger foundations and a major contributor to the recognition of 
psychology as a scientific discipline. Basic principles of 
psychometric theory led to the development of respected intelligence 
tests and large and comprehensive testing and assessment programs. 
This paper synthesizes major developments in psychometric theory, 
focusing on the issues of reliability and validity. The synthesis is 
developed from: (1) Cattell's basic data relation matrix (BDRM) of 10 
coordinates for describing psychological events and the reliability 
and generalizability (Cronbach) coefficients derived from it; (2) the 
four box conception of data (predictor, criterion, experimental 
treatment, and nonexperimental treatment) to depict psychological 
research problems; and (3) a hierachical version of Brunswik's lens 
model in which correlations between boxes can only be optimal under 
conditions of good reliability and symmetry. The paper concludes with 
an empirical example from intelligence and school achievement 
research: predicting school grades and other kinds of aggregated 
criteria from the Berlin model of intelligence, the mcst prominent 
intelligence model in German-speaking Europe. (bS) 
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Psychoflietrlc theory has been one of psychology's stronger foundations and has 
contributed a lot toward recognizing psychology as a scientific dIsclpllne.Be- 
ginning with the London School ISpeannan , 1904, see Stanley 197) or Eysenck, 
1981) basic principles of psychometric theory led to the development of re- 
spected Intelligence tests and gave nvj to large and coBiprehenslve testing 
and assessment programs especially In the United States. Reliability and vali- 
dity Issues iCronbach, 1984a| attracted an uncounted nagnltude of researchers 
and led to a uiyrlad of publications* 

yet In the last decade psychowtrlc theory was the focus of harsh criticism 

iLunsden, 197G). Weiss S Davison (1981) stated: 
"Somewhere during the three-quarter century history of CTT (classical 
test theory, Inserted by the author) the real purpose of reliability 
estimation seems to have been lost. Reliability coefficients In and 
of themselves have little utility for practical situations except for 
comparing their nagnltudes In order to justify the use of measuring, 
Instruments/ |p.633| 

Yalldltyi especially construct validity also was often labeled as 'confusing" 

(Cronbach» 1985). 

In every science there Is a time for analysis and a time for synthesis, In 
psychometric theory time Is ripe for synthesis. 

A comprehensive theory should synthesize most of the major developments up to 
the present. It Is not niy Intention to deemphaslze contributions by eminent 
researchers by not mentioning them. But In my eyes, when talking about rell- 
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ability and validity, the pillars of the psychometric mansion are Caltell's 
(1966) basic data relation matrix (BDRHI, Cronbach et al. (19721, general- 
Izabinty theory and Brunswik's (1956) representative design. 

If we try to understand change and validity better than through paradoxes, 
take Cronbach's (1957, 19751 complaints about the two disciplines o^ scien- 
tific psychology seriously, and really want to go "beyond the two disciplines 
of scientific psychology' we have to Integrate another pillar of the research 
methodology edifice, namely that erected by the so-called Northwestern School 
(Campbell, Cook, Boruch. see Glass, 19831. 

BORH AND GENERALIZABILITY THEORY. 

Cattell (1966, p.78| claimed that every psychological event Is complet:!; 
described by embedding It In the KHlmenslonal BDRM. These 10 coordinates are: 
(II person. (2) stimulus, (31 response, W situation-occasion, (5) observer, 
(61 states of the person, (71 variants of stimuli, (81 styles of response,(9| 
phases of the environmental background, (101 conditions or states of the ob- 
server. Cronbach (1984 b| In honoring Cattell acknowledged the Influence of 
the BdrH In developing general Izablllty theory. In Cronbach et al's. (1972) 
terminology the coordinates are facets of generalization which are partitioned 
according to ANOVA principles. Cattell (19661. using for reasons of simplicity, 
only three coordinates also partitions the data box, thus demonstrating dif- 
ferent factor analytic techniques. Fig, 1 shows the "unfolding", comparable to 
a computer-printout according to a three dimensional data box consisting of 
coordinates, persons, variables and occasions. 

Insert Fig. 1 about here. 

Such a data box contains only one observation per cube cell, but Is easily 
conceivable as a replicated data box, containing more than one observations 
per cube cell. One can always argue how many coordinates are sufficient for 
a concrete research problem. I found it most essential that every researcher 
have at least a hunch about what s/he wants, what s/he does not want, and what 
'Is simply random error. Every BDRH is decomposable into variance / covariance 
"between wanted" facets (bw). "between unwanted" facets (bu) and "within wanted 
and unwanted" facets (wuwl of generalization. 
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For BDRH's Mlth one observation per Intersection of xanted and uniianted facets, 
It Is not possible to Isolate the Interaction wanted x unwanted [m] from the 
randon error (e) variance. But this Is possible for replicated BDRN'S.I.e. with 
more than one observation per Intersection. 

Unrepllcated BDRH; 
<"^tot'fbw*^b«*t^ 

The syibol for set union means, that In this case Interaction and random error 
are confounded. 
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'''^bw^^bu^V ' 
'''^bw^^w«u^V ' 



' ' tot bw bu e 
How Is reliability tied to the BDRH? This Is very easy. Reliable variance Is 
true variance. True variance Is systeiutlc variance* 1*e. not random error var* 
lance, Thereforewe add over all sources of systenatic variance, I.e.: 

^troe'^bw^^bu^^xu 

Sm over these variance / covarlance matrices and sum over total matrix, to get 
the true and total variance of the first ccntrold. Then simply divide the two 
to get the reliability coefficient for the first centrold of such a BDRH: 
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Overall reliability Is always the sum of reliability for wanted, unwanted, and 
the Interaction wanted x unwanted coordinates or facets. 
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Generallzablllty coefficients are easily derlvable.Ulth these coefficients we 
want only to generalize over the wanted, unwanted , Interaction or combinations 
of these facets. The base for comparing generallzable variance 1s then only 
wanted or unwanted variance etc., plus random error variance: 
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Generallzablllty coefficients are Incommensurable and cannot easily be summed 
up, whereas our former coefficients are. Hultlvarlate reliability coefficients 
are easily developed using Cohen's (19821 set correlatlor system (Ulttiann, In 
press). 

The BDRH as In Fig. 1 can be partitioned accordlngtootherficets than persons. 
This leads to the well known techniques of factor analysis or In generallzablllt 
theory, to what Cardlnet, Tourneur and Allal (1976) have demonstrated. 

The Cardlnet", Tourneur- and Allal -symmetry means developing general Uabllity 
coefficients for the other possible partltlonlngs of the BDRH. 

Reliability is best defined as a set of answers to questions of repeatability. 
This Is nowhere more visible than In the partitioning o' the BDRH. 

What is won in knowing these reliability or generallzablllty coefficients ? 
Reliability is a prerequisite for validity and you have to make sure that the 
wanted aspects of your research program are repeatable. What should be done 
when the magnitude of your wanted reliability coefficient or component is not 
high enough ? Host often we use the Spearman-Brown prophecy formula and try 
to lengthen our tests. If we do It practically we use aggregation (Epste1n,l9B3) 
How does aggregation work 1 
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Substituting Eq. (51 in the Spearmati-Broim prophecy fonnula £q. (13) gives 
after some manipulation: 
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or for wanted variance: 
115) r. 



tt. 



He must always be sure to have parallel tests for our wanted components. 
These tests should be as heterogenous as possible with respect to unwanted 
variance. 

Eq.(rj) can be suraiariied as follows: Reliability conceived as a general 
concept of science depends on variability of wanted, unwanted and error 
facets. He can influence reliability through 

ID Choosing greater heterogenity with respect to wanted variance: 
\\ 1. Huniphreys (1952, 1979) has always nade this proposal. 

12) Greater hoiiiogenity of unwanted variance, arrived at through 
cancellation of different heterogenous components, to decrease 

13) Increasing the test k times with perfect parallel tests, and thus 
decreasing ly.l'C^^jindl'c^l by factor k. 

14) HlniBizing the ratio of unwanted to wanted variance I'lC, +C 11 / 
pr I bu wxu' 

' ''bw'* 

So far nothing has been said about validity, though often, falsely in my 
eyes, generilizabillty to a universe of content Is related to validity (i.e. 
construct validityl. The relationships between reliability and validity can 
best be conceptualized with four separate data boxes. 



THE FOUR BOX CONCEPTION 

Four data boxes are necessary to depict most of psychology's research problens. 
The first box is a predictor box. Box number 2 is the criterion box. Box 
nupiber 3 is an experimental treatment box, where persons are randomly gsslgned 
to planned treatments. Box 4 is the nonexperimental treatment box, containing 
treatments where randomization is not possible or containing unplanned treat- 
ments with positive or negative influence for criterion boxes. Fig. 2 gives 
a pictorial representation of the four box conception. 

Insert Fig. 2 about here. 

it is iranediately evident that questions of reliability for the predictor box 
and the criterion Lox can be asked,although we will also recognize that 
questions of reliability of criteria are not so much emphasized as the reli- 
ability of predictors. Partitioning leads to reliability for the predictor 
box: 
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and for the criterion box 
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As a first result we can develop a canonical correlation corrected for attenu- 
ation with these reliabilities, i.e. for the first canonical copnent of 
wanted variance; 

il/2 

1181 Cancor (pr,cr)^ . '-^ 'Fi cr) 



V 



w 



This reasoning liiiiediately leads to the question of reliability of a planned 
(etrboxl or an unplanned (ntr-boxl treatment. How can one develop a formula 
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for it ? In ANflW notation m pjrtltlon total variance {^^^^) in variance 
betwen Ibql and within groups Uql. To obtain a reliability formla for an 
etNbox w divide the systematic Itruel variance (bql by total variance, i.e.! 
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However, within groups, all subjects are assigned the same score (either one 
or zero when using dum^ coding). Thus the variance within is alwjj's a zero 
Mtrix and the reliability of an experimental treatnent box is always one I 

In the ideal experiient it is assuoed that through proper randomization and 
operational ization each subject receives the same amunt of treatment or no 
treatnent. In reality however, the variability within groups can be enormous 
and the reliability of the planned treatment approaches zero. Treatment as 
Intended has no effect then, i.e. does not correlate with criteria. F.e. a 
modification of eq. (18), i.e. 

^etr,cr ' ""tf ««> 
w 

or so low, thus attenuating a true cancor (otr.cr) so much, that it does not 
reach significance. 

This does not mean that variability within groups cannot correlate with cri- 
teria, this is an Independent possibility. To Illuminate this problM further, 
we can compare it to Hurray's concept of e<- and /3. press, 

Regressing boxes on each other can be done via set correlation. Cohen (1982; 
Cohen i Cohen, 1983) have already developed, prepared and sharpened that 
tool. 

Regressing criterion boxes on treatment boxes and predictor boxes is the route 
experimentalists prefer.This arrangement of the "four-box-concept1on-map"shows 
the Northwestern passage. Campbil), Cook, Boruch et al. frwi Northwestern Uni- 
versity recomended it over all these years as the best research design f.e. 
in evaluation research if this randomization is feasible. Cronbach (19821, in 
the same area, preferred the Southwestern passage • regressing criterion boxes 
on nonexperlmental treatment and predictor boxes, with the assuqitlon of ob- 
taining more generallzable results for real world problems. 



The Implicit continuum In Fig, 2 between etr-box and ntr-box can be regarded 
as the gradual flow from true experiments to quasl-experlments. 

Relating these four boxes means trying to answer questions of validity, which 
Include prediction and explanatlon.To obtain successful prediction and ex- 
planation we urgently need a principle of syimetry. 

'imm SY^ETRT) PRINCIPLES FOR SUCCESSFyi PREDICTION AND EXPLANATION. 

In science, principles of symmetry often lead to new discoveries and solution 
of long standing problems. Brunswlk's lens model has this built In beauty. I 
have used a hierarchical version of the lens model to Illustrate how success- 
ful prediction works (Ulttmann, In press). Fig. 3 Illustrates how predictor 
boxes and r: iterlon boxes should look If we use hierarchical (f.e. personality) 
models. Only syimetrlcal relationships are fair tests of validity. 

Insert Fig. 3 about here. 

An unfair test is f.e. relating a broad secondary factor with a single act 
behavioral criterion either in personality or aptitude research and hoping for 
a high correlation. Correlations between boxes can only be optimal under con- 
ditions of , ,od reliability anisymietry. The boxes have to contain syuetrlcal 
components. In terms of the lens model equation (Tucker, 19(4) the correlation 
between boxes f.e. pr and cr is: 



(20) r, 



pr,cr ' V.cr VclJi^^pr.crVi^ sj]^ 



Vi cr correlation between linearly predictable pr-boxes and linearly 
predictable cr-boxes. and R^^. are ^ ;lple correlation coefficients mapping 
the linear predictability of pr or cr components, respectively. 

So far nothing In Eq. 20 Is said about reliability. G can be attenuated as 
every coefficient of validity. ' 
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,true 
'pr,cr 
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, which means 
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Sr.cr' ^ "''tlonshlp, Is attenuated by the iquare root 

of the product of the reliability coefficients of wanted variance In predictor 

and criterion box. Substituting eq. (22) In eq. (20) and dropping the non- 
linear ten for convenience and clarity, gives: 

£q.(23) shoMS how a true correlation can be attenuated. First through lack of 
reliability of the predictor box variables, second through lack of reliability 
of criterion box variables, third through lack of construct reliability of 
the predictor box variables and forth thnusli lack of construct reliability 
of the criterion box variables (or construct Indicators). 

I prefer to label Rp^. or 9,^^ as construct reliability, meaning the amount of 
overlap of our indicators tilth Intended or wanted constructs. The ten vali- 
dity should be reserved for relating different constructs. 

Boruch i Gomez (1977) have proposed overlap Indices between Intended and 
actually measured aspects of treatment and response variables. Leinhardt i See- 
wald (1981) discuss In Instructional research such overlap between what Is 
taught and what Is tested as a crucial point In evaluating curricula, in the 
lens inodel equation these overlap indices already have a numerical solution. 
I wst quickly aphasize that these Indices are only realistically measurable 
or estimable if we know what we want. Should we want to measure a construct 

at a secondary or a primary level, how large or narrow should the breadth 
of the construct be ? Hore general constructs have a lower family resemblance 
of constituting elements (Wittgenstein, 1953) than constructs at a lower level 
of generality. The popular prototype approach ts' In danger of loosing breadth, 
leading possibly to constructs of a lower level. Overlap or construct rell- 
»bility can be lowered by too few or by too many Indicators for a wanted con- 
struct. In the first case relevant indicators are missing, In the second case 
unwanted Indicators are Included. Remember that asyimetry is spetrlcal In 
both directions I Under ideal circumstances R -R « 1 should hold and the 



same must be true for all other palrwise relations between boxes. But piy 
attention also that this does not automatically tell you at what level of 
generality you are. 

We could draw many Interesting relationships between all four boxes with 
Insights In measurements of change and how most paradoxes of classical test 
theory vanish under such an approach (see Wittmann, In press). But there 
Is no place to denonstrate all that here. 

Let us conclude with an empirical example from Intelligence and school 
achievement research, i.e. relating the pr-and cr-boxes. 



PREDICTING SCHOOL GRADES AND OTHER KINDS OF AGGREGATED CRITERIA FROM THE 
BERLIN HODEL OF INTELLIGENCE. 

The Berlin model of Intelligence (JSger, 1982, 1984) nowadiys Is the most 
prominent model concerning structure of Intelligence In german speaking 
Europe. A short description of the model and how It had be«n developed by 
principles of systematic aggregation Is given In Wittmann (in press). The 
model has as many psychometric models a hierarchical structure. At the 
bottom 48 test are grouped In four factors of an operation mode and three 
factors of a content mode. The content modelCN)factors are Number INCH), 
Figure (FCN) and Verbal (VCN). The operation mode (OP) contains aggregates 
of more process-like aspects of Intelligence: 
Speed on tasks (SOP), Hemory (HOP), Creativity (COP) and Processing capaci- 
ty for complex Information (POP). These seven factors constitute a kind of 
primary factor level. The two modes as broader factor classes can be re- 
garded as a secondary level and g-lntelllgence as an aggregate over all sevei 
factors at the top of the pyrailde. We are already accustomed to laying the 
hierarchy on the side In the lens model framework. Fig. 4 shows this model. 

Insert Fig. 4 about here, 

Perception Imiedlately forces us to look for a symmetric structure at the 
criterion side (cr-box) and to ask what kind of criteria we can predict with 
such a model. Admittedly we do not know. But JSger (1982) in addition to 
measuring his model components, had gathered school grades, Interests, and 
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fictofwiljftleilly itrUa tcilti of ttlf-iviluittd ibilititi in dlffirefit 
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Kith tNM wbox viriiblii m cm try diffirintly igjregited virliblis to 
find fm Mlikh livil of gwirillty thiy in bitt ixpliinid. In nliting 
dlffirtdt tilt of virliblit froi pr-ind er-bom m utid iit-comlitlDn 
ICoInn, m\ CoHm t Cohin, 1983) it Mitlvirliti cornlition ind In 
rilitlflg tlngli virliblit or tingli iggngilid Indlcit trit^ leti of virl* 
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not Iflcludid) Mfi It foIIONt. 
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with thi til of four opiritlvi ind thru contint fictors (flPIN); 
"s. WIN ' l'^'^" 

Pirtlll tit cornlitlont of ichool gridit with contint factors, operitivi 
fictort pirtlilid out of iich tit: 
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pirtlil tit comlitlon of Khool gridis with opir«tl»i factors, content 
fKtort pirtlilid out of iich iit: 

icN,OP.CN'''«« 

Thiti tit cornlitlont Indlciti that thi Birlln nodel is a good predictor of 
tchool gridit (at m htvi knoM for i long tiM frai good IntelHgince tests). 
fw thi pirtpictlvi of differintlil villdlty, thi content factors show a 
highir validity thin thi opiritlvi fictort. Hi do not know Nhy,but cm spicu- 
liti that through thi viry proceti of grading, opiratlve factors an iggn- 
gitid out,dlilnlthid or thiti iptUudis an not gradid. Regarding the restric- 
tion of ringi with ritpict to Intilllgenci In a 'GyMailw'-saivIe all corre- 
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latloni are levenly undereitiinated. 
Table 1 shows thi rijult of single gradii, ;otiI (cumulative) grade and 
two nion cofliplex composites using multiple regnssion. Operative factors 
(OPFM) and content factors (CNfAC) art correlated. Ue used comionallty. 
analysis jCooley i Lohnes, 1976) to orttiogonalize both factor classes In 
unique OPW iu{p), In unique CNFAC (u[^) and the cownality between the 
two tits (Cqpji,), Ui now tee mon clearly the predictive validity of each 
level of gtnerillty regressed on these cr-varlables. CN-factors have th« 
Mghest coefficients, although in some grades then is an advantage of OP- 
factors. Interwtlngly this is most obvious In politics/history. Separate 
analysis for that gnde showed that creativity (COP) was the single best 
pndlctor. 

The three highest validities wen found for the two complex composites and 
grades In mathematics. The numbers in parenthesis are computed according 
to HcNenar (1949,p.I26)under the very conservative assumption that the sample 
of "Gpaslum'-puplls an restricted In range canpared to the total popula- 
tion of m with respect to standard deviation. Approximately 20J of an 
age cohort visit "Gymnasium" in Germany, 

Insert Table I about hen. 

The two composites AGGNAUl and AGGEIMI represent European philosophers 
beloved distinction between "Naturwissenschaften" and "Geisteswissenschiften" 
(sH Legend Tab. 1), 

Obviously most syiietrical to the seven factors of the Berlin intelligence 
model 1$ a complex compound of achlevemnt in science classes, interests and 
personality. g-intel1lgence also predicts this Index best, though to i MCh 
SMlIer degne than the seven factors. Total average grade Is the third best 
predicted criterion from g and here we see no difference! after considering 
shrinkage) compared to the prediction froai the seven factors. 

Very astonishing is the jero correlation of g with AGGEIUI. 

Tab. 2 a and b show the hienrchical regnssion of AGGNAUI and AGGEIUI on 
the seven factors. In Tab. 2 a.we see that Number (NCN) correlates with a 
different sign than Verbal (VCN). Thus aptitudes cancel each other and the 
lero cornlation becoMs explainable. He also see that Number (HCN) and 
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Figure (FCNj are suppressor variables. 

Both suppress irrelevant variance in creativity ICOP), Partialed creativity, 
which mans creativity in ninnerical and figural tasks partialed out.ls the 
best predictor of AGGEIMl. This partialed creativity is than best described 
as verbal creativity. Suppressors always give hints that a more syiimetrical 
relationship Is obtainable at a lower level of generality where Irrelevant 
lunMRtedl conponents m removed. Looking at AGGNAWl we need at least four 
froi the seven factors In an additive linear equation to explain the cri- 
terion variable. These factors sten froni both modes, i.e. processing capa- 
city (POP), speed on tasks ISOPI from the operation node and number (NCN) 
and figure |FCN) from the content mode. 
The results are embarassing to the beloved distinction mentioned above 
between "naturwissenschaftllche" and "gelsteswissenschaftliche" educational 
achievements. One can often hear that the latter are more differentiated and 
need more complex aptitudes constellations, in contrast to the more "narrow- 
minded" science adherents, advocates who only deal with numbers and figures, 
from the point of view of this exploratory analysis quite the opposite seems 
to be true. 

1 hope 1 was able to whet your appetite for these concepts and at the same 
time convince you that the synthesis proposed really is an improvement of 
construct and predictive validity. 
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Tablei^i Predictive validities for various criteria from the Berlin model of 
Intelligence factors. 



Grades 

Geman 

English 

2nd foreign 
language 

Mathematics 

Chem/Blo/- 
Physlcs 

Politics/ 
History 

Arts/Hus1c 

Sports 



Total ave. 
grade 



AGGNAHI 
AGGEIHI 



(.478) 
.2207 
(.321) 

.1165 
(.173) 

.4513 
(.604) 

.3532 
(.493) 

.2847 
(.407) 

.2063 
(.302) 
.1774 
(.261) 

.3309 
(.466) 

.6048 
(.752) 

.4267 
(.578) 



R' 


R' 

OP 


11' 

CN 


G 


OP 


"CN 


""OP CN 


.1165 


.0420 


.0998 


.0233 


.0167 


.0745 


.0253 


Mil 


.013/ 


.0460 


.0088 


.0027 


.0350 


.0110 


.0136 


.0121 


.0012 


.0000 


.0124 


.0015 


-.0003 


.2037 


.1583 


.1747 


.1156 


.0290 


.0454 


.1293 


.1247 


.1198 


.1085 


.1011 


.0162 


.0045 


.1036 


.0792 


.0661 


.0357 


.0240 


.0435 


.0131 


.0226 


.0426 


.0170 


.0124 


.0069 


.0302 


.0256 


.0132 


.0315 


.0262 


.0150 


.0133 


.0165 


.0053 


.0OB7 


.1095 


.0913 


.1017 


.0861 


.0078 


.0182 


.0835 


.3658 


.2151 


.3104 


.1354 


.0554 


.1507 


.1597 


.1821 


.0476 


.1639 


.0084 


.0182 


.1345 


.0293 



Lejend: AGGNAMI Is a composite of grades In Mathematics, Chemlstry/Blology/- 
Physlcs, Interests In vocational areas of natural sciences and self- 
evaluated abilities In these classes and areas. 
AGGEIHI Is a composite of grades In LanguagesiGerman, English, se- 
cond foreign language). Interests In Arts, Humanities, and self- 
evaluated abilities In these domains. ' 
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UtrtrlUtni VARIABU 




AGGEim 




Vtrlible 


Kultjple R 


II Square 


RSQ Change 


Simple R 


COP 


.21198 


.04494 


.04494 


(.018) 


-.21198 


NOP 


.21278 


.04528 


.00034 


(.002) 


-.01843 


SOP 


.21300 


.04S37 


.00009 


(.000) 


.00960 


POP 


.21808 


.04756 


.00219 


I.OOO) 


.04681 


NCN 


.3S192 


.12384 


.07628 


(.033) 


.18288 


FCN 


.42562 


.18115 


.05731 


(.000) 


-.00347 


VCN 


.426$; 


.16205 


.00090 


(.130) 


-.36115 


b) 


DEPENOT VJUtlABLE 




mm 




Variable 


Hultlple R 


R Square 


RSQ Change 


Simple R 


COP 


.10899 


.01188 


.01188 


(.000) 


-.10899 


HOP 


.13912 


.01935 


.00747 


(.015) 


-.08646 


SOP 


.19496 


.0381 


.01866 


(.037) 


-.13659 


POP 


.4(378 


.21509 


.17708 


1.002) 


-.42081 


NCN 


.S8395 


.34100 


.12590 


(.263) 


-.51303 


FCN 


.60350 


.36422 


.02322 


(.038) 


-.19493 


VCN 


.60479 


.36578 


.00156 


(.009) 


.09576 



Table 2 ; Hultlple Regression analysis with Jiger's seven Intelligence 
factors on two coirplexly aggregated Indices, 

legend ; Factor scores m used for prediction - operative factors first, 
then content lactors. The numbers In parentheses give the squared 
multiple correlations with content factors first and operative 
fKtors second In a hierarchical regression analysis. 



ERJC 18 




Figure 1 : Unfolding the three-dimensional data bu 
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Planned randomized experimental treatments 
S treatment Intensity and treatment Integrity 



Predictor variables ^ 



/ Criterion variables 




J Predictor Side 



/ ntr-box 

/ Treatments not randomized and 

/ positive or negative effects 

criterion variables 



unplanned Influences with \ 
on treatment outcomes,1.e* 



unfair test 



secondary 
factor level 



primal 
factor' levi 



Criterion Side 



fair test (symmetrical) 
primary level 




ictor level 
pr,cr 
\ 

fair symmetrical test 
at secondary level 



secondary 
factor level 



Flaure 2 : The four box conception 



F^a^^^g 3 : Hierarchical version of Brunswik's lens model for denoting 
principles of symmetry between predictors and criteria. 



20 



21 



BERLIN INTELLIGENCE MODEL COMPONENTS AS PREDICTORS 



PROCESSING 
CAPACITY 
(POP) 




Systematic aggrega- 
tion over the 



seconda 
factor level 



primary 
factor 



operation and the 
content modes 



using suppressor 
principles 



at the single 
test level 



primary 
factor level 
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• 
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o 
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POTENTIAL CRITERIA 



SCHOOL GRADES 
INTERESTS 

SELF-EVALUATED ABILITIES 

VARIANTS OF 

AGGREGATED 

INDICES 



I 



'secondary 
level ???? 



''^9"''^ ^ • The Berlin intelligence model in ]ens model framework 
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